Digital camera with integrated accelerometers

ABSTRACT

A digital camera system has integrated accelerometers for determining static and dynamic accelerations of the digital cameral system. Data relating to static and dynamic accelerations are stored with recorded image data for further processing, such as for correcting image data for roll, pitch and vibrations and for displaying recorded images with a predetermined orientation using information about, e.g., roll. Data may also be used on-the-fly for smear suppression caused by vibrations.

CROSS-REFERENCE TO RELATED APPLICATION

This is a divisional application of application Ser. No. 121624,451,filed on Nov. 24, 2009, which is a divisional application of applicationSer. No. 12/472,478, filed on May 27, 2009, which is a continuationapplication of application Ser. No. 10/847,354, now U.S. Pat. No.7,554,578, filed on May 18, 2004, which is a continuation-in-partapplication of application Ser. No. 09/897,435, now U.S. Pat. No.6,747,690, filed on Jul. 3, 2001, which claims priority under 35 U.S.C.119(e) to Provisional Application Ser. No. 60/217,023 filed on Jul. 11,2000, the entire contents of which are hereby incorporated by referencein their entirety for all purposes.

FIELD OF INVENTION

The present invention relates to a digital camera system havingintegrated accelerometers for determining static and dynamicaccelerations of said digital camera system. Data relating to thedetermined static and dynamic accelerations are stored with recordedimage data for further processing, such as for correcting image data forroll, pitch and vibrations. Data may also be used on-the-fly for smearsuppression caused by vibrations.

BACKGROUND OF THE INVENTION

When using rectangular film formats like the 35 mm format, images arerecorded on film with a “landscape” (horizontal) orientation in respectto the common way of holding a camera. When the photographer wishes tocapture a portrait he will tilt the camera 90 degrees and thus acquirean image with a “portrait’ (vertical) orientation. Later when thedeveloped images are viewed, the viewing person will manually orientthem correctly. Since the images are on paper, it is relatively easy toreorient some of them.

In digital photography the landscape orientation is the default settingfor most cameras. When the captured images are viewed on a display, theywill appear with a landscape orientation with no respect to whether theimages were actually captured with the camera held in a portrait orlandscape orientation. The images then have to be manually inspected andlater possibly rotated to reflect their original orientation. Somedigital camera manufacturers are now beginning to include a sensor unit,which detects whether the camera is placed in landscape or portraitposition when an image is captured.

In U.S. Pat. No. 5,900,909 an orientation detector which consists of twomercury tilt switches is described. The two mercury switches make itpossible to determine whether the user is holding the camera in thenormal landscape orientation or in a portrait orientation. There are twoportrait orientations: One is the result of a clockwise rotation whereasthe other is the result of a counter clockwise rotation. The use ofmercury switches has some distinct disadvantages in that mercury cancause great damage when it interacts with the human body, and for thatreason it is quite unpopular in many products. Mercury switches usuallyconsume a lot of space in comparison with monolithic IC's. This is dueto their very mechanic structure, which makes miniaturisation difficult.In a digital camera it is crucial to minimise the size and weight, so inrespect to this, the use of mercury and other primarily mechanicallybased switches, is not the optimum choice. A mercury switch basedsolution in a digital camera is limited to detecting a few roughorientations, i.e. landscape and portrait. The robustness and ease ofuse of the mercury switch are its primary advantages today.

The main limitation regarding micro-mechanical accelerometers fabricatedin e.g. silicon is related to their ability to absorb shock withoutbeing damaged.

Taking pictures with long shutter times and maybe even a high degree ofzoom makes the image capture process very sensitive to vibrations, whichwill result in blurred images. At short shutter times the image is lesslikely to be affected by vibrations since most vibrations, which willaffect a camera, have an upper frequency limit, due to mechanicaldamping from the surroundings. Especially handheld photography easilyresults in blurred images when longer shutter speeds are used. Onesolution to the described limitations is to be able to compensate formost vibrations. Vibrations can be compensated optically by means of alens module, which is capable of moving the projected image around inthe image focus plane. This requires a special and expensive lens.

When vibrations cannot be compensated, another way of helping thephotographer to acquire the optimum images is to inform him about anypossibility of blurring, which may have occurred in a captured image.With feedback from the camera regarding the degree of shaking during theexposure time, it is possible for the photographer to decide whether hewants to capture another image of the same scene.

In U.S. Pat. No. 4,448,510, a camera shake detection apparatus isdescribed. It includes an accelerometer, which is connected to a controlcircuit, which activates an alarm, when the acceleration exceeds acertain predefined threshold level. The threshold level can beinfluenced by the exposure time—a long exposure time results in a lowthreshold level and vice versa for a short exposure time. The outputfrom the accelerometer may also be forced through an integrator beforecomparing the output to a threshold level to account for the fact thatblurring is more probable to occur if a large number of highaccelerations are detected. None of the described implementations areable to determine if the camera after a short period of vibrationsreturns to its initial position or the position where the majority ofthe exposure time has been spent. In such a case the suggestedimplementations would generate a “blur” alarm, even though the imagecould be sharp.

In some applications, especially the more technically oriented, it canbe an advantage to have knowledge about how the camera is physicallyoriented in space. In a set-up with a digital camera connected to a GPSreceiver, knowledge about the roll and pitch of a camera can be used toautomatically pin point the scene being photographed. This can be usedin aerial photography and other related technical applications. In otherset-ups, feedback to the photographer about the exact roll and pitch canbe useful for him to correct his orientation of the camera. Another useof the roll information is to automatically correct for small degrees ofslant in the sideways direction. In most common photographic situationsit is not desirable to have an automatic correction of a slight slant,as the photographer often wants full control of the image orientation. Afeature like automatic slant correction should be user configurable inthe sense that it can be turned off and on.

JP 58-222382 discloses an apparatus that automatically correctsinclination of scanned originals by changing the address where the imagedata is written to reflect the original with no inclination. Inclinationis measured by using feedback from a couple of timing marks, which areconnected to the slant of the original. Measuring the inclinationthrough the use of timing marks is not useful in digital stillphotography. General image rotation in software is carried out by movingthe original image data to a new position in another image file/buffer.

The present invention may be implemented in a digital still camera or adigital still camera back and supply a total solution which is verycompact, consumes little power, and is applicable in a variety ofdigital still camera applications. The use of a single detector unit fora variety/plurality of functions decreases the physical size, lowers thepower consumption, and keeps the price down. The use of amicro-mechanical accelerometer as opposed to a mercury switch has thedistinct advantage that it does not contain mercury.

The micro-mechanical accelerometer has several advantages over themercury switch and the pendulum based orientation detector. Some ofthese advantages are:

-   -   it can easily be miniaturised,    -   it is a measurement device with a high degree of accuracy which        can be configured dynamically for a variety of applications        through the use of different processing which can be integrated        in a digital processing unit or analogue electronics,    -   it may be applied to measure both static and dynamic        acceleration at the same time. In comparison, the mercury switch        and the pendulum are both optimised for measuring static        orientation.

With the integration of more than one measurement axis in asilicon-based chip it becomes possible to measure both dynamic andstatic acceleration in several directions at the same time. The staticacceleration is basically obtained by low-pass filtering the raw outputsfrom the accelerometer(s). More sophisticated filtering can be appliedto handle specific requirements. With static acceleration from at leasttwo axes—which are perpendicular to each other—it is possible to obtainthe precise degree of both roll and pitch for a digital still camera.This may be used in technical applications for automatic or manualcorrection of slant in both sideways and forwards directions. Mercuryswitches or pendulums are limited to a more rough evaluation of theorientation of the camera (basically limited to two positions).

A subset of the before-mentioned static acceleration measurement featureis the possibility to automatically determine when an image should bedisplayed with portrait or landscape orientation. The high precision ofthe roll and pitch information makes it possible to determine thecorrect orientation under the most difficult conditions where a slightmechanical tolerance for a mercury switch or pendulum based solutioneasily would result in an unexpected determination of orientation.

The mercury switch and pendulum switch based solutions lack thepossibility to be dynamically configured to each users need, as theirfunctionality is fixed mechanically when they leave the factory. Anexample of this could be a user who wishes that his camera shoulddisplay images with a landscape orientation until he tilts the camera 75degrees, whereas the normal configuration would be to display an imagewith a portrait orientation when the camera is tilted more than 45degrees.

The measurements of dynamic acceleration (vibration) during the time ofexposure may be used in a variety of ways to reduce the possibility ofthe photographer taking a blurred image. The use of active compensationfor camera movements can be used to extend the previous working rangefor photography in terms of longer exposure time, more zoom, and theability to capture images in vibration dominated surroundings, i.e.helicopters.

With a traditional film camera it is necessary to have an expensive lenswhich corrects the induced vibrations by changing the optical path ofincident light. When the vibrations are compensated either by plainimage processing with input from the recorded movements, or by activecompensation through movement of charges in the image sensor, or byphysically moving the image sensor itself, all the outlined compensationsolutions described in detail below, enable the use of any type of lens,and are still able to reduce blur. The addition of a little extra imageprocessing to compensate for vibrations through post-processing, or theuse of charge movement in the sensor, does not increase themanufacturing cost, as opposed to a solution which changes the opticalpath.

When using accelerometers, generation of a “blur” warning is much morefail safe than earlier solutions which were not able to determine if thecamera after a short period of vibrations would return to its initialposition or the position where the majority of the exposure time hadbeen spent. In such a case the earlier implementations would generate a“blur” alarm, even though the image could be sharp.

SUMMARY OF THE INVENTION

The present invention is therefore directed to a digital still camerawhich substantially overcomes one or more of the limitations anddisadvantages of the related art. More particularly, the presentinvention is directed to a digital still camera with a sensor unit fordetermining static and dynamic accelerations, and methods thereof whichsubstantially overcomes one or more of the limitations and disadvantagesof the related art.

It is an object of the present invention to provide a sensor unit todigital cameras which is very compact, consumes little power, and isapplicable in a variety of digital camera applications.

It is a further object of the present invention to provide a sensor unitto digital cameras capable of providing the following features:

-   -   Low-pass filtering the accelerometer outputs enables exact        measurement of roll and pitch which can be used in technical        applications for automatic or manual correction of slant in both        sideways and forwards directions. The roll and pitch information        is also useful in applications where knowledge of the camera        shooting direction is needed, i.e. aerial photography.    -   A subset of the before mentioned feature is the possibility to        automatically determine when an image should be displayed with        portrait or landscape orientation.    -   A processing unit evaluates the raw accelerometer outputs during        the time of exposure, The processing unit evaluates whether or        not the measured vibrations may result in an image, which        appears to be blurred, The photographer may receive a warning in        case the processing unit finds that blur is highly likely to        occur in the captured image.    -   The raw accelerometer outputs can also be used to keep track of        the movements of the camera with respect to the field of        gravity. When the image is processed afterwards it is possible        to correct the image for blur by using the record of camera        movements during the exposure time. During the exposure time,        the camera movements can be actively compensated by moving        charges (pixel information) in the image sensor in a direction        to follow the movements of the projected image in the image        plane. The movement of charges in the image sensor can be        combined or replaced with mechanical actuators to physically        move the image sensor.    -   In some cases a little blur may be advantageous to reduce the        amount of Moiré image defects which may be introduced when an        image is extremely sharp. Using the knowledge about the camera        movements during the time of exposure it is possible for the        image processor to generate an image with less tendency to show        Moiré without the full reduction of sharpness.    -   A processor receives at least static acceleration data to        continuously or at short intervals evaluate the camera's present        orientation in comparison with a pre-set orientation, and to        indicate a difference between these.

In a first aspect, the present invention relates to a sensor unit to adigital camera, said sensor unit includes a detector which determinesstatic and dynamic accelerations. The detector includes, a first sensorwhich senses acceleration in a first direction, and provides a firstoutput signal in response to acceleration in the first direction; and asecond sensor which senses acceleration in a second direction andprovides a second output signal in response to acceleration in thesecond direction, the second direction being different from the firstdirection. The sensor unit also includes a processor which processes thefirst and second output signals. The processor includes a first filterwhich low-pass filters the first and second output signals so as toobtain information relating to static accelerations, and a second filterwhich band-pass filters the first and second output signals so as toobtain information relating to dynamic accelerations.

The first and second directions may be perpendicular to each other. Thesensor unit may further include a third sensor which senses accelerationin a third direction and provides a third output signal in response toacceleration in the third direction, the third output signal beingprovided to the processor so as to obtain information relating to staticand dynamic accelerations. The third direction may be perpendicular tothe first and second directions.

The sensor unit may further include an alarm, which may generate analarm signal in response to at least one of the output signals from thesensor. The alarm signal may be generated when at least one of theoutput signals exceeds a predetermined level which may relate to thefact that an image starts to get blurred or relate to a certain amountof exposure time. The alarm signal may be constituted by a sound signal,a flashing signal, an image file tag or any combination thereof.

At least one of the sensors may include a micro-mechanical deflectionsystem. The first, second and third sensors may be integrated in asingle micro-mechanical deflection system mounted in the camera house ofthe digital camera—for example in a digital camera back.

At least one of the above and other objects may be realized by providinga method of determining static and dynamic accelerations in a digitalcamera, the method including:

-   -   providing a first sensor sensitive to acceleration in a first        direction, said first sensor being adapted to provide a first        output signal in response to acceleration in the first        direction,    -   providing a second sensor sensitive to acceleration in a second        direction, said second sensor being adapted to provide a second        output signal in response to acceleration in the second        direction, the second direction being different from the first        direction,    -   low-pass filtering the first and second output signals so as to        obtain information relating to static accelerations, and    -   band-pass filtering the first and second output signals so as to        obtain information relating to dynamic accelerations.

The method may further include providing a third sensor sensitive toacceleration in a third direction. The third sensor provides a thirdoutput signal in response to acceleration in the third direction, thethird output signal being provided to the processor so as to obtaininformation relating to static and dynamic accelerations.

The first, second and third directions may be essentially perpendicular.The method according to the second aspect may further include generatingan alarm signal as mentioned in relation to the first aspect of thepresent invention.

At least one of the above and other objects may be realized by providinga digital camera including

-   -   an image recording device, the image recording device comprising        a plurality of light sensitive elements,    -   a first translator which translates the image recording device        in a first direction in response to a first input signal,    -   a sensor unit according as set forth above, wherein the        band-pass filtered first output signal from the first sensor is        provided as the first input signal to the first translator so as        to compensate for determined dynamic accelerations in the first        direction.

The digital camera may further include

-   -   a second translator which translates the image recording device        in a second direction in response to a second input signal, the        second direction being different from the first direction,    -   a sensor unit as set forth above, where the band-pass filtered        second output signal from the second sensor is provided as the        second input signal to the second translator so as to compensate        for determined dynamic accelerations in the second direction.

The first and second directions may be essentially perpendicular. Thefirst and second translators may translate the image recording device ina plane substantially parallel to a plane defined by the plurality oflight sensitive elements. The first and second translators may comprisemicro-mechanical actuators.

At least one of the above and other objects may be realized by providinga method of processing image data, the method including:

-   -   providing image data, the image data being stored in a memory,    -   providing data or information relating to static accelerations        as described above, providing data or information being recorded        and stored with the image data, and    -   correcting the image data in accordance with the data or        information relating to static accelerations so as to correct        the image data and reduce the influence of roll and pitch.

Alternatively, the roll and pitch information may be used to determinewhether the optimum way of displaying the image is with a portrait orlandscape orientation. At least one of the above and other objects maybe realized by providing a method of correcting image data duringrecording of an image of an object, the method including:

-   -   recording image data of the object by projecting the object onto        an array of light sensitive elements, recorded image data being        generated as electrical charges in the array of light sensitive        elements,    -   providing information relating to time dependent movements of        the array of light sensitive elements relative to the object,        and    -   correcting the recorded image data in accordance with the        provided information relating to movements of the array of light        sensitive elements relative to the object by moving charges        (pixels) in the array of light sensitive elements so as to        correct for relative movements between the array of light        sensitive elements and the image of the object.

At least one of the above and other objects may be realized by providinga method of displaying a recorded image with a predeterminedorientation, the method including:

-   -   providing information relating to the degree of roll of the        recorded image, the information being provided by first and        second sensor means sensitive to accelerations in a first and a        second direction, respectively, the second direction being        different from the first direction, and    -   using the provided information to determine the orientation by        which the recorded image is to be displayed and/or stored.

The orientation by which the recorded image is to be displayed and/orstored may comprise portrait and landscape orientations. The user maydetermine at which predetermined acceleration levels the recorded imagetoggles between portrait and landscape orientation. The predeterminedacceleration levels may correspond to a predetermined degree of roll ofthe recorded image.

At least one of the above and other objects may be realized by providinga method of correcting image data during recording of an image of anobject, the method including:

-   -   recording image data of the object by projecting an image of the        object onto an array of light sensitive elements,    -   providing information relating to time dependent movements of        the array of light sensitive elements relative to the image of        the object, and    -   correcting the recorded image data in accordance with the        provided information relating to movements of the array of light        sensitive elements relative to the image of the object by        counter moving the array of light sensitive elements so as to        compensate for the time dependent movements.

At least one of the above and other objects may be realized by providinga method of reducing Moiré image defects without full reduction insharpness, the method including:

-   -   providing an array of light sensitive elements,    -   recording an image of an object using the array of light        sensitive elements, the image being affected by movements of the        array of light sensitive elements relative to the object so that        the recorded image appears to be blurred and without Moiré        defects,    -   providing information relating to time dependent movements of        the array of light sensitive elements relative to the object        during the time of exposure, and    -   using the provided information as an input to an image        processing algorithm so as to reduce Moiré image defects in the        recorded image and thereby obtain a modified image with        increased sharpness.

At least one of the above and other objects may be realized by providinga computer program including code adapted to perform the methodaccording to the any of the above methods when the program is run in acomputer. The computer program may be embodied on a computer-readablemedium.

BRIEF DESCRIPTION OF THE DRAWINGS

The present invention will now be described with reference to theaccompanying figures, where

FIG. 1 shows a digital still camera system, where the digital back isoptional;

FIG. 2 shows roll and pitch of a digital camera with respect to thefield of gravity;

FIG. 3 shows a block diagram of a digital camera;

FIG. 4 illustrates two monitoring axes, where the x-axis is used tomonitor the pitch, and 30 the y-axis is to monitor the roll;

FIG. 5 shows the pitch working range;

FIG. 6 shows the roll working range;

FIG. 7 shows an original image (left) and the image after correction(right);

FIG. 8 shows how images, which are captured under different pitch androll conditions, will be displayed;

FIG. 9 illustrates how the imaging sensor can be moved in one or moredirections in the imaging plane using piezo elements or other exactmicro-positioning devices;

FIG. 10 illustrates how charges may be moved up or down two rows at atime to match a color filter pattern;

FIG. 11 shows moving of the imaging sensor horizontally using a singlepiezo element or other micro-positioning device, and moving the pixelsin the imaging sensor vertically;

FIG. 12 shows the camera orientation function embodied by a processor;and

FIG. 13 shows a visual indicator that indicates changes in orientationof the camera towards or away from the stored orientation.

DETAILED DESCRIPTION OF THE INVENTION

In the following description, for purposes of explanation and notlimitation, specific details are set forth in order to provide athorough understanding of the present invention. However, it will beapparent to one skilled in the art that the present invention may bepracticed in other embodiments that depart from these specific details.In other instances, detailed descriptions of well-known devices andmethods are omitted so as not to obscure the description of the presentinvention with unnecessary details.

The digital still camera system as shown in FIG. 1, where the digitalback is optional, incorporates a section which is able to determine theroll and pitch of the camera with respect to the field of gravity, seeFIG. 2. The same section also monitors the vibrations, which occurduring the time of exposure. A block diagram can be seen in FIG. 3. Thesensor section is comprised of one or more accelerometers, whichmonitors acceleration in two or three axes placed perpendicular to oneanother. Together with a digital and/or analogue signal processingsection it is possible for the camera to recognize both staticacceleration (e.g. gravity) and dynamic acceleration (e.g. vibration)through the use of the same accelerometer unit(s). Preferably theaccelerometers are in the same IC. The digital still camera systemconsists of a lens, a camera house, and in some cases of a digitalcamera back which is attached to the back of the camera house. Thesensor section may be placed anywhere in the digital still camerasystem.

Preferably the accelerometer(s) are of the micro-machined type which isintegrated in or on a monolithic structure. There are several ways toimplement a micro-mechanical accelerometer. One is to form a cantileverin silicon with a very small thickness (pm range). When the entirestructure of the device shakes or moves quickly up and down, forexample, the cantilever remains still due to its inertia so that thedistance between lever and a reference layer changes correspondingly.Such changes in distance between lever and reference layer may be sensedin terms of corresponding changes in electrostatic capacitance betweentwo electrodes, where one is connected to the lever and the other to thereference layer.

Another principle uses piezo-resistors on the surface of the cantileverbeams and their resistance is stress dependent. Acceleration causes abending of the cantilever beams, which causes stress. Using twolongitudinal and two transverse piezo-resistors, which have oppositesigns of resistance changes, and connecting them to a Wheatstone Bridgemakes it possible to get a signal voltage which is proportional to theacceleration.

For yet another type of micro-electromechanical accelerometer the sensoris a surface micro-machined structure built on top of the silicon wafer.Polysilicon springs suspend the structure over the surface of the waferand provide a resistance against acceleration forces. Deflection of thestructure can be measured by using a differential capacitor, whichconsists of independent fixed plates, and central plates attached to themoving mass. The fixed plates are driven by 180° out of phase squarewaves. Acceleration will deflect the beam and unbalance the differentialcapacitor, resulting in an output wave whose amplitude is proportionalto acceleration. Phase sensitive demodulation techniques are then usedto rectify the signal and determine the direction of the acceleration.The output of the demodulator is low pass filtered with a cutofffrequency, which sets the measurement bandwidth limit. A simple digitaloutput signal can be obtained by letting the filtered output drive aduty cycle modulator stage.

One or more accelerometers which monitors two or three axes, which areperpendicular to one another, may advantageously be mounted in a digitalstill camera system. With the accelerometer(s) it is possible todetermine both the roll and pitch of the camera with respect to gravitywith a very high degree of accuracy.

When the accelerometer(s) is mounted with monitoring axes as shown inFIG. 4, the x-axis is used to monitor the pitch, and the y-axis is tomonitor the roll. Using two axes, the camera movements can be monitoredcorrectly as long as the camera is not upside down—the working range forboth roll and pitch is a 180° rotation, which is most commonly used inphotography. FIG. 5 shows the pitch working range and FIG. 6 shows theroll working range of a 2-axis system. With a 3-axis system, which alsouses information from the z-axis, it is possible to achieve 360° rolland pitch rotation. The degrees of roll and pitch are preferablyobtained during the time of exposure and after the accelerometer outputtypically has been heavily low pass filtered to prevent aliasing due tohandshake, i.e. if the accelerometer which is being used containspre-processing circuits that transforms the analogue output(s) from thebasic sensor unit to digital output(s), it is in general mostadvantageous to use digital signal processing techniques to define therequired measurement bandwidth, since it is easier to adapt and optimizefor various shooting conditions in terms of varying exposure time andvibrations in the environment surrounding the shooting scene.

The roll and pitch information is very accurate and can be used asfeedback to the photographer to help him physically orient his cameracorrectly to obtain images without sideways or forwards slant, i.e.,pendulum and mercury tilt sensors are not usually able to accomplishthis without being physically very large, which makes them unsuited fordigital still cameras. The photographer may choose to use a piece ofpost-processing software which automatically corrects a slight sidewaysslant in the image by rotating the image counter wise a certain amountof degrees, which is equivalent to the roll information that wasrecorded during the time when the image was captured. Finally the imagemay be automatically cropped to fit the frame. FIG. 7 shows an example.

Since both roll and pitch are measured, the photographer also has accessto information about the pitch of the camera, and is thereby able tocompensate for this manually or through the use of post-processingsoftware. Knowledge about both sideways and forwards slant can beadvantageous in many technical applications. The roll and pitchinformation, which is acquired during the time of exposure, is eitherembedded in the image file format or attached to a standard image fileformat. When the image file is displayed, the display software or apre-processing algorithm can utilize the accurate roll and pitchinformation to determine the proper orientation of the image and displayit either as a portrait or landscape picture. Hysteresis on the rollmeasurement is used to prevent unexpected switching between portrait andlandscape display modes. See FIG. 8, which shows how images which arecaptured under different pitch and roll conditions will be displayed.The rough sideways rotation can be correctly determined in just aboutany situation—even when the camera is a couple of degrees from pointingstraight to the ground or straight up in the air. if the pitch of thecamera shows that the photographer is shooting straight up in the air orstraight to the ground, it doesn't make sense to use the rollinformation to determine how the image should be displayed, instead theimage is displayed in landscape, which is most often the naturalorientation of a camera image plane. This eliminates the possibility ofunexpected rotation of the image when displayed. Without the describedcheck on the pitch reading, images which are captured with the camerapointing straight up or down with almost the same physical orientationmay be displayed with different orientations. This is sometimes the casewhen using pendulum or mercury based tilt sensors.

Using an image sensor, which enables readout of pixels from each cornerin two directions, it is possible to rotate an image without the use ofa large temporary storage media (RAM), that way relieving systemresources and reducing the overall system overhead. Image information isread straight from the image sensor, which will result in an image withthe proper rough orientation (landscape, portrait clockwise, andportrait counter clockwise) as determined by the roll and pitchinformation which was stored during the time of exposure.

The roll and pitch information can be updated continuously or regularly(e.g. several times per second) to inform the photographer about thepresent orientation of the camera. This feature has a number ofapplications.

A first application of the continuously updated orientation informationis as an electronic spirit level which can help the photographer tocapture images which are perfectly aligned with the horizon. The rolland pitch information can be presented to the user in various ways. Theabove outlined procedure can also be used to help the photographercapture images that are aligned with plumb objects.

A second useful application is to have a memory function equivalent toe.g. man-over-board functions of Global Positioning Systems where thesystem guides the user back to a previous position. When a user recordimages with the camera having a given orientation, the roll and pitchinformation, which is acquired during the time of exposure, is stored.When, at a later stage, the user wants to restore the photographicset-up of the previous recording, recalling the stored roll and pitchinformation will allow the system to guide the user to position thecamera with the same orientation. The roll and pitch information fromthe previous recording may be stored in a file related to the previouslycaptured image data, or may, upon activation of an orientation memoryfunction at the previous recording, be stored in a dedicated orientationmemory. This function may be applied e.g. when making a series of imagesof an object from different positions or at different points in time,when recording time lapse movies as well as when shooting movingpictures, where the shooting angle needs to be kept constant betweendifferent takes.

The applications related to the continuously updated orientationinformation of the camera may be embodied as shown in FIG. 12. Thecamera orientation function is embodied by a processor illustrated andcontrolled by the camera orientation menu. The processor can receivecontinuously updated information relating to static accelerations fromanother processor (or from another program controlled by the sameprocessor) receiving input from the two accelerometers. The receivedinformation relating to static accelerations corresponds to the presentorientation of the camera in roll and pitch.

Upon activation of the ‘Store present orientation’ functionality, thecamera orientation function stores present roll and pitch information inthe memory. Upon activation of the ‘Recall stored orientation’functionality, the camera orientation function obtains stored roll andpitch information from the memory, and correlates these data with thepresent roll and pitch information to give an indication to the userwhich guides the user to orient the camera. The indication is given viathe Audio and/or Visual indicator. Upon activation of the ‘Spirit level’functionality, the camera orientation function correlates the presentroll information with a pre-set roll value corresponding to horizontalorientation.

The camera orientation function is adapted to correlate the updatedorientation information with the stored orientation information, and cangenerate correlation signals indicating a relative difference betweenthe updated and stored information. Typically, there will be correlationsignals relating both to the roll orientation and to the pitchorientation.

The indicator generates an output based on the correlation signals fromthe camera orientation function. This output is adapted to guide a userto orient the digital camera so that the updated (i.e. present)orientation information is at least substantially equivalent to thestored orientation information. The output is adapted in that itindicates a continuously or regularly updated difference between thepresent orientation information and the stored orientation information,so that, when the user changes the orientation of the camera, the outputchanges in a manner so that the user understands whether he/she changesthe orientation of the camera towards or away from the storedorientation. The indicator may be a visual indicator such as a graphicalillustration such as shown in FIG. 13 shown on a LCD on the camera.Another visual indicator is a mechanical device equivalent to what isknown from gyroscopes in airplanes. An audio indicator may be afrequency or amplitude modulator connected to a speaker. The modulatorincreasing the frequency or amplitude of a sound as the camera system isgetting closer to being perfectly aligned with the predeterminedorientation.

The accelerometer(s) serve double duty, as their output(s) are alsobeing used to determine the vibrations (dynamic acceleration) whichoccur during exposure. Vibration information is basically obtained usingthe raw accelerometer output or maybe by applying some high or band-passfiltering of the output(s) from the accelerometer(s). The filter can beboth analogue and digital, typically with the digital filter as thesmallest and with the ease of adaptability.

Vibrations during the exposure time will blur the captured image, andare therefore usually unwanted. The image is most sensitive tovibrations when the exposure time is relatively long or when thephotographer zooms in heavily. Whether or not the vibrations, whichoccur during exposure, will affect the final image depends upon thenature of the vibrations. If the camera is placed in the same steadyposition for 99.9% of the exposure time, and shakes severely for theremaining 0.1% of the exposure time, the final image will not lookblurred. Whereas an image will look blurred when it has been capturedwith the camera in the same steady position for 50% of the exposuretime, and the remaining 50% of the exposure time the camera isphysically slightly offset from its initial position. The point is thathigh acceleration can be accepted for a short amount of time (in respectto the exposure time) as long as the camera returns to its originalposition, or the position where the majority of the exposure time hasbeen spent.

Naturally the photographer would prefer that vibrations are removed bymechanical means, but in some cases, i.e. handheld photography, it isnot possible. Another way to reduce/remove blur is to monitor themovements of the camera during the exposure time and compensate for themovements by either moving the image which is projected on the imageplane or by moving the imaging sensor.

The vibration information from the accelerometer axes during theexposure time can be used as feedback to reduce the blur in the capturedimage. Information about acceleration over time along with informationabout the optics, which generates the image in the imaging plane, willenable blur to be removed/reduced in many ways. The following describedmethods can be used individually or in combination with one another.

-   -   Using the knowledge about how the projected image moves around        in the imaging plane over time, it is possible to mathematically        reconstruct the original image by calculating “backwards” from        the final image. This solution requires a total log of measured        accelerations from the accelerometer(s) axes.    -   The imaging sensor can be moved in one or more directions in the        imaging plane using piezo elements or other exact        micro-positioning devices, see FIG. 9. Thus, it will try to        follow the way the projected image moves around in the imaging        plane. A solution with two piezo elements takes up quite a bit        of space, is expensive, and uses quite a bit of power.    -   The charges (pixels) in the image sensor can be moved up and        down to follow the movements of the projected image in the        vertical direction. This method has some distinct advantages, in        that it does not consume any considerable amount of power and        does not take up any space. Unfortunately it is limited to the        vertical direction. If an image sensor with a Bayer colour        filter pattern is used, charges will have to be moved up or down        two rows at a time to match the color filter pattern, see        FIG. 10. With a monochrome sensor charges can be moved one row        at a time.    -   A combination of moving the imaging sensor horizontally using a        single piezo element or other micro-positioning device, and        moving the pixels in the imaging sensor vertically, see FIG. 11.        This combination makes it possible to follow the projected image        in both the horizontal and vertical direction at a lower cost,        lower power consumption and using less space than a solution        which incorporates two piezo elements.

The vibration pattern is analysed during the exposure cycle. If theacceleration exceeds a certain level for a certain amount of time, whichis determined in respect to the exposure time as described in theearlier example, the photographer will receive a warning, which isvisual and/or audible and/or attached to the image data. The vibrationwarning may be automatically turned off by the camera when a flash lightis used, since the duration of a flash light burst is very short (<1ms), thereby reducing the possibility of vibrations during the time whenthe majority of the light from the exposure hits the imaging sensor.

In most cases where an image is slightly blurred, the image can beimproved by applying a sharpening algorithm to the blurred image. Withthe vibration information at hand, it is possible for the camera toautomatically apply an optimum amount of sharpening to a blurred image.Sharpening can be used as an automatic stand-alone module, which can beadded to the resulting image from the before mentioned methods, whichall contribute to reduce blur in the image.

In certain cases a little vibration of the camera may be advantageous asit reduces the possibility of Moiré artifacts in the captured image dueto the induced blur. Again using the information about the movements ofthe projected image in the imaging plane, will enable the imageprocessing software to produce a developed (processed) image with lesstendency to show Moiré artifacts without the full loss of sharpness.

It will be obvious that the invention may be varied in a plurality ofways. Such variations are not to be regarded as a departure from thescope of the invention. All such modifications as would be obvious toone skilled in the art are intended to be included within the scope ofthe appended claims.

1. A method of correcting forward slant in an image recorded by adigital camera with a predetermined orientation, said method comprising:providing a first accelerometer integrated on a monolithic chip in thedigital camera, the first accelerometer being sensitive to accelerationin a first direction lying in an imaging plane of the digital camera,said first accelerometer being adapted to provide a first output signalin response to acceleration in the first direction; providing a secondaccelerometer integrated on a monolithic chip in the digital camera, thesecond accelerometer being sensitive to acceleration in a seconddirection lying in an imaging plane of the digital camera and beingdifferent from the first direction, said second accelerometer beingadapted to provide a second output signal in response to acceleration inthe second direction; providing a third accelerometer integrated on amonolithic chip in the digital camera, the third accelerometer beingsensitive to acceleration in a third direction normal to the first andsecond directions, said third accelerometer being adapted to provide athird output signal in response to acceleration in the third direction;low-pass filtering the first, second, and third output signals to obtaininformation relating to static accelerations; using the informationrelating to static accelerations to identify a pitch angle of thedigital camera when the image was recorded; and correcting forward slantin the recorded image in accordance with the identified pitch angle toreduce the influence of pitch away from a vertical orientation or ahorizontal orientation.
 2. The method according to claim 1, furthercomprising using the information relating to static accelerations toidentify a roll angle of the digital camera when the image was recorded;and wherein the forward slant correction also utilizes the identifiedroll angle.
 3. The method according to claim 2, further comprisingcorrecting sideways slant in the recorded image in accordance with theidentified roll angle to reduce the influence of roll away from thevertical orientation or the horizontal orientation.
 4. The methodaccording to claim 2, wherein the forward slant correction comprisesperforming a keystone correction or a perspective correction of therecorded image based on the identified pitch and roll angles.
 5. Themethod according to claim 1, wherein the forward slant correction iscarried out by post-processing software.
 6. The method according toclaim 1, wherein the forward slant correction is carried outautomatically.
 7. The method according to claim 1, wherein said first,second, and third output signals are provided as continuous functions ofaccelerations in the first, second, and third directions.
 8. The methodaccording to claim 1, wherein providing said first, second, and thirdaccelerometers comprises using first, second, and third micro-mechanicalaccelerometers as the first, second, and third accelerometers.
 9. Themethod according to claim 1, further comprising cropping the correctedimage to fit a rectangular frame.
 10. A digital storage holding softwareconfigured to perform the method of claim 1 when executed by a processorin the digital camera.
 11. Post-processing software for correction offorward slant in an image recorded by a digital camera and beingconfigured to perform the following when executed by a processor:obtaining a first output signal in response to acceleration in a firstdirection lying in an imaging plane of the digital camera; obtaining asecond output signal in response to acceleration in a second directionlying in an imaging plane of the digital camera and being different fromthe first direction; obtaining a third output signal in response toacceleration in a third direction normal to the first and seconddirections; low-pass filtering the first, second, and third outputsignals to obtain information relating to static accelerations; usingthe information relating to static accelerations to identify a pitchangle of the digital camera when the image was recorded; and correctingforward slant in the image in accordance with the identified pitch angleto reduce the influence of pitch away from a vertical orientation or ahorizontal orientation.
 12. The post-processing software according toclaim 11, wherein the correction of forward slant in the image can beturned off and on.
 13. A digital camera providing correction of sidewaysslant of images, the digital camera comprising: an image sensor a firstaccelerometer integrated on a monolithic chip in the digital camera, thefirst accelerometer being sensitive to acceleration in a first directionlying in a plane of the image sensor, said first accelerometer beingadapted to provide a first output signal in response to acceleration inthe first direction; a second accelerometer integrated on a monolithicchip in the digital camera, the second accelerometer being sensitive toacceleration in a second direction lying in a plane of the image sensorand being different from the first direction, said second accelerometerbeing adapted to provide a second output signal in response toacceleration in the second direction; a third accelerometer integratedon a monolithic chip in the digital camera, the third accelerometerbeing sensitive to acceleration in a third direction normal to the firstand second directions, said third accelerometer being adapted to providea third output signal in response to acceleration in the thirddirection; a processor being configured to process the first, second,and third output signals, said processor comprising a low-pass filterfor filtering the first, second, and third output signals to obtaininformation relating to static accelerations and to use the informationrelating to static accelerations to identify a pitch angle of thedigital camera; and means for automatically correcting forward slant inimages in accordance with the identified pitch angle to reduce theinfluence of pitch away from a vertical orientation or a horizontalorientation.
 14. The digital camera according to claim 13, wherein themeans for correcting forward slant in the recorded image can be turnedoff and on.
 15. The digital camera according to claim 13, furthercomprising a digital storage and wherein the means for correctingforward slant in the recorded image comprises software held in thedigital storage.
 16. The digital camera according to claim 13, whereinthe processor is further configured to use the information relating tostatic accelerations to identify a roll angle of the digital camera, andwherein the means for automatically correcting forward slant alsoutilizes the identified roll angle.
 17. The digital camera according toclaim 16, further comprising means for automatically correcting sidewaysslant in images in accordance with the identified roll angle to reducethe influence of roll away from the vertical orientation or thehorizontal orientation.
 18. The digital camera according to claim 16,wherein the means for automatically correcting forward slant comprisesmeans for performing a keystone or a perspective correction of therecorded image based on the identified pitch and roll angles.
 19. Thedigital camera according to claim 13, wherein said first, second, andthird accelerometers are first, second, and third micro-mechanicalaccelerometers, and wherein said first, second, and third output signalsare continuous functions of accelerations in the first, second, andthird directions.
 20. A method of correcting keystone or perspective inan image recorded by a digital camera, comprising the steps of providingoutput signals indicative of acceleration from first, second, and thirdorthogonal accelerometers integrated on a monolithic chip in the digitalcamera; low-pass filtering the output signals to obtain informationrelating to static accelerations; using the information relating tostatic accelerations to identify a pitch and roll angle of the digitalcamera when the image was recorded; and performing a keystone or aperspective correction of the recorded image based on the identifiedpitch and roll angles.